A Concise Introduction to Receiver Operating Characteristic (ROC) Curve

ROC curve analysis is used widely in medicine as a method for evaluating the performance of diagnostic tests (3,5,6,10), but has been used recently in many agricultural applications (2,4,5,11,12). The ROC curve provides information regarding how often a test’s predictions are correct, and provides a graphical method for evaluating and discriminating between different diagnostic tests or modifications of the same test (12). To perform the analysis, one starts with a diagnostic test that produces a range of values or test scores (T) in which a classification is decided. Decisions are arrived at by comparing the diagnostic’s output to a threshold value (Tthresh). The data (i.e., individuals or test subjects) are then partitioned into two groups: the ‘cases’—in which the disease is known to occur; and the ‘controls’—in which disease was absent. Those test subjects with test scores above the threshold (T>Tthresh) are classified as diseased (D+), and those with test scores equal to or below the threshold (T#Tthresh) are classified as not diseased (D-), irrespective of their true disease status. For various reasons, diagnostic tests are not perfect predictors of disease. This can be depicted graphically as two overlapping distributions of threshold values; the cases and controls (Figure 1)(5,10). Thus, any decision threshold based on this test yields one of four possible decisions: 1) true positive (TP), in which a ‘case’ was correctly classified as diseased; 2) true negative (TN), in which a ‘control’ was correctly classified as not diseased; 3) false positive (FP), in which a ‘control’ was incorrectly classified as diseased; and 4) false negative (FN), in which a ‘case’ was incorrectly classified as not diseased (6,7). Now, say N individuals (test subjects) were classified under the rules of the diagnostic test, X of the N subjects were classified as ‘cases’, and Y of these as ‘controls’. Then the true positive proportion (TPP) is the number of true positive decisions divided by the total number of cases. TPP is referred to as the ‘sensitivity’ of the test. The true negative proportion (TNP) is the number of true negatives divided by the number of controls. This is referred to as the ‘specificity’ of the test. The false positive proportion (FPP) is the number of false positives divided by the number of controls. The false negative proportion (FNP) is the number of false negatives divided by the number of cases. In probabilistic terms, TPP is an estimate of Prob(T>Tthresh | D+) (read as ‘the probability of a test score above the threshold, given the presence of disease). Similarly, FNP is an estimate of Threshold Fr eq ue nc y